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1. Introduction 

Graph expansion methods for statistical models have been extensively discussed in the 
literature. They have been very helpful in studying the high-temperature behaviours 
and the phase transition properties of discrete models such as the Ising model of 
ferromagnetism. Some of the early efforts were carried out by Brout and others [U El E] . 
More recently, Georges and Yedidia [I] found that, high-temperature expansion of the 
Ising spin glass free energy can also be carried out under the constraints of fixed mean 
spin values. This later work was extended by Sessak and Monasson [Sj to include pair 
correlations of spin variables as another set of expansion constraints. The constrained 
graph expansion method was applied to the inverse Ising problem [5], with the aim of 
inferring the microscopic interactions of a Ising system based on the observed mean spin 
values and spin-pair correlations. 

For finite-connectivity binary statistical models, Chertkov and Chernyak [61 [7J 
showed that the partition function can be expressed as a sum of contributions from 
subgraphs. The first term of this expansion is identical to the partition function 
obtained using the Bethe-Peierls (BP) approximation. Loop corrections to the BP 
approximation was also calculated by Montanari, Rizzo, and collaborators [HI E] and by 
Parisi and Slanina [10J. The derivation of the partition function expansion by Chertkov 
and Chernyak relied on a special property of Ising spin variables (see equation (17) 
of [7]). This special property is not valid for more general statistical models, whose 
microscopic variables are not necessarily binary or discrete. Whether the conclusion 
of [6l [7] is valid to general statistical models is still an open issue (for models whose 
discrete variables take Q > 2 values, a complicated loop-tower expansion was presented 
in HU). 

In the present contribution we first extend the results of [6j [7J [11] by carrying out 
a very simple derivation of partition function loop series for a general statistical model 
defined on a graph. We do not make any assumptions on the nature of the microscopic 
state variable of each edge (or vertex) of the graph. This state variable can be discrete 
or real-valued, or be a vector, or even be a function itself. We show that the first 
term of this expansion is also identical to the Bethe-Peierls (BP) partition function, and 
corrections to the BP partition function come only from looped subgraphs without any 
dangling edges. The auxiliary probability distributions of this loop series expansion are 
chosen to be a fixed point of the belief-propagation equation. This particular choice 
makes all the subgraphs with at least one dangling edge to have zero contribution to 
the correction terms. 

As the second main result, we present the loop series expressions for the grand 
partition function and higher-level partition functions. The belief-propagation equation 
of a statistical model may have multiple fixed points, each of which is referred to as a 
macroscopic state of the configuration space. If this happens, we define a grand partition 
function at the level of macroscopic states and perform a loop series expansion for this 
grand partition function. When the auxiliary probability distributions of this expansion 



Loop series for general statistical models 



3 



are chosen to be a fixed point of the survey-propagation equation (first derived through 
the first-step replica-symmetry-breaking (1RSB) mean-field theory of spin glasses |12j). 
the first term of this loop expansion gives the BP free energy functional at the level 
of macroscopic states. Corrections again come only from subgraphs that are free of 
dangling edges. In case the survey-propagation equation has multiple fixed points, 
the same loop series expansion can be performed for higher-level partition functions. 
As a result, we obtain the higher-level BP free energy functionals and the correction 
contributions, and the associated message-passing equations. 

This work is a mathematical approach to the theory of spin glasses from the 
framework of partition function loop expansion. It is clear that at each replica- 
symmetry-breaking (RSB) level of the mean-field theory [12] the corrections to the 
free energy due to looped nontrivial subgraphs are neglected. This neglected correction 
contribution is explicitly expressed as a logarithm over a finite series in this paper. 
At a given level of macroscopic states we anticipate that, the magnitude of the total 
loop correction contribution to the free energy will be sub-linear in N (N being the 
total number of vertices) if there is only one fixed-point for the corresponding message- 
passing equation, but it will be linear in N if there exist multiple fixed-points. This 
statement needs to be checked by numerical calculations on single graphical systems. 

Section [2] introduces the general statistical model. We work out the loop series of 
the partition function in Sec. [3] and derive the belief-propagation equation. In Sec. [4] 
we extend the discuss to the case that the belief-propagation equation has multiple 
fixed points, and perform a loop series expansion for the grand partition function. The 
1RSB survey-propagation equation is derived here. We conclude this work in Sec. [5j and 
discuss some possible extensions. The |Appendix A contains graph expansion results for 
a one-dimensional ring. 



2. General statistical models on graphs 

We consider a graph G composed of N vertices {i = 1, 2, . . . , N) and M edges. An 
edge between two vertices i and j has a state variable Xy. This variable may be a 
binary spin for some systems, Xij = ±1. For other systems, may be real- valued or 
be a vector, or be even more complicated. In this paper we make no assumptions on 
the nature of the microscopic state variable x%j of each edge Each vertex i has 

an energy Ei(x m ), where x idi = {x ih ,x ij2 , x ijk } with j u j 2 , ...,j k being the k other 
vertices with which % forms an edge. The number k of nearest neighbors of a vertex 
might be different for different vertices. The vertex energy is a function of the state 
variables of its connected edges. Notice that x^ and Xjt both denote the state of edge 
therefore x^ = Xji. The total vertex energy for an edge configuration {x^} is 

N 

E({x tJ }) = Y,E l (^)- (1) 

8=1 
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e -0Ei(x igi ) 



The partition function of the system is defined as 

r N 

Z(/3) = I][ dx^^jjn^^. (2) 

(id) <=i 

In the above equation, (3 is the inverse temperature, po(xij) is the probability of 
microscopic state Xij for an isolated edge and Ylnj) means the product over all 

the edges of graph G. For simplicity we assume that the a priori probabilities po(xij) 
are identical for all edges. This assumption is of cause nonessential. 

The partition function ^ also applies to graphical models whose microscopic states 
are defined on vertices rather than on edges [6], [7] . For example, consider a graph G 
with the property that its vertices can be divided into two subsets, denoted by {i} (the 
variable nodes) and {a} (the check nodes), such that all the edges of G are between a 
variable node % and a check node a. For each variable node i we assume that 

dx iPo (xi) J] f X ™, f* ], (3) 

aedi P0\Xia) 

where di denotes the set of nearest neighboring check nodes of i. Equation (]2j then 
simplifies to 

Z W) = U[[ dxipoix,)] JJ e -W»*O f (4) 

i J a 

where da denotes the set of nearest neighboring variable nodes of a. Equation Q is 
the partition function of a system defined on a factor graph, with each variable node % 
having a microscopic state and each check node a having an energy E a . The check 
energy E a depends on the microscopic state XQ a of the variable nodes in da. 

In some graphical models, the state x^- of each edge is a collection of two 

microscopic states y\j and yjj, x^ 

= {UijiUij}- We assume that the a priori probability 
distribution of the edge state x^ equals to Po(ylj)Po(Vij), an d that the energy E { of a 
vertex i can be expressed as 

e~ 0E ^ = f dytpoMe-Mto^V R f^'^ } • (5) 

J jedi poyyij) 

Under these assumptions, the partition function ^ becomes 

Z(^) = l[[Jdy i p (Vi)e-^ iiyi ' V9i) ], (6) 



which describes a graphical model whose vertex energy Ei depends on the microscopic 
state yi of vertex i and the microscopic states of its nearest neighbors. An example 
of such statistical models is the palette-coloring problem [TjU dH [15] . 

3. Graph expansion for the general statistical model 

To find a loop series expression for the partition function (|2]), we introduce for each edge 
two auxiliary probability distributions qj^i(xij) and q^j{xji), and rewrite Z(f3) as 

m = n n [/ ^1]^ n f^ftl (7) 

i=ljedi L J qk^i{Xik)qi->k{Xki) 
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(id) 



N 

n n [j dx iji3- 



Xi 



-/3Ei(x m ) 



n[i+A 



(k,l)(Xkl, Xlk, 



(fc,0 



In (1st) , C(i,j) is an edge constant with the value 



Xijjqj. 



(9) 



and A(i t j)(xij, Xji) is expressed as 

( X ji) Qj-^i \ X ij ) 

From ^ we know that the partition function can be expressed as the sum of 
contributions from all the possible non-empty subgraphs of G: 



A 



\ x ij > x ji ) 



1. 



(10) 



ZW) 



'BP 



gee 



In the above equation, Z BP is calculated by 

rii=i Iljfedi J dxijqj^i(xi 



'BP 



-/3Ei(x idi ) 



(12) 



A non-empty subgraph g of graph G contains a partial set of the edges of G and all the 
vertices that are attached to these edges. The correction L g is expressed as 



n 





J dXjjijjf—^Xjj) 


e -/3Ei(x idi ) 


Hjedi 


Id'/,//, .;(//..;) 


e -/3Ei(y igi ) 



(k,l)eg 



(13) 



Consider a subgraph g which has a vertex z that is linked to the other parts of g 
only through a single edge (i, j). The neighborhood of such a leaf vertex i is shown 
schematically in figure [TJ We find that after integrating over the variable x^, the 
correction L g is expressed as 

-f3E k (x hdk ) 



n 



ledk 



n n 



J dx k iqi^ k (x k i) 



fdy M qi^ k (y, 



id, 



Q—0Ek(ykdh) 



n 



(m,n) (Xmni X nm 



(m,n)£g\(i,j) 



X 



qi^j{xji) J dyijqj^i(yij)qi^j 

(yij)/po(yi. 



where q 
through 



{ Qi^>j( x ji) I dyijqj^.i(yij)qi-).j 
is determined by the set of probability functions qg^j 



Qi^>j( X ij) — Bi^>j(q, 



di\j, 





J dx ik qk^i(x ik ) 


e 


/3Ei(x m ) 


JdyijPoiyi^UkmM 


Jdy ik q k ^i(y. 


k) 


e -PEi(y i8i ) 



(14) 

{q k ^i, k E di\j} 



(15) 



The function Bi^.j(qgi\j) as defined by (15) is called the belief-propagation equation. It 
takes as input a set of probability distributions q k ^i(xi k ) {k 6 di) and outputs a new 
probability distribution q^^Xij). 

Since we are free to choose the auxiliary probabilities functions {qi-+j(xji)}, we 
can choose this set of auxiliary functions to be a fixed point of the belief-propagation 
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Figure 1. The neighborhood of a vertex i. A solid line between any two vertices i 
and j means that the edge is presented both in graph G and in subgraph g. A 
dashed line between two vertices i and k means that the edge (i, k) is presented in G 
but not in g. A solid circle denotes a vertex that belongs to subgraph g, and a dashed 
circle denotes a vertex that is not belonging to g. A solid circle is attached by at least 
one solid edge. In this figure, vertex i is connected by only one solid edge, it is a leaf 
vertex of subgraph g, and the edge is a dangling edge. 



equation (15). In other words, we require that the auxiliary probability functions to 
satisfy 

qi^j{xij) = Bi^j^qk^Xik), k e di\j}). (16) 

Then for each edge we have &_^(a:y) = q^j^Xij), and the expression inside 

the curly brackets of (14) is identically zero. Under this special choice, only those 
subgraphs of graph G with each vertex % having at least two attached edges have non- 
zero contributions to the correction of the partition function. The total free energy F(/3) 
is then expressed as 



1 



F{p) = --\nZ{fi) 



1 



F BP (/3)--li, 



(17) 



1+ E W 

g'QG 

where g' denotes a looped subgraph that contains no dangling edges. The free energy 
F BP ((3) corresponds to the partition function Z BP and is expressed as 



with 



F B p{P) 



fi 



i (id) 



kedi 



f(i,3) = ~ p 111 



dxi 



dxikqk 



/3Ei(x igi ) 



(18) 



(19) 



(20) 



We emphasize that Fbp(/3) is identical in form to the mean-field free energy as 
obtained by the replica-symmetric (RS) spin-glass theory [12]. The expression (18) was 



first derived in the mean-field theory by using the BP approximation. The free energy 
F BP can also be viewed as a functional of the 2M probability distributions {p^j^x^)} 
on the M edges of graph G. In this paper we refer F B p as the BP free energy 
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functional. It is easy to check that the variation of Fbp with respect to any a probability 
distribution p. 



is zero at a fixed point of (16). Equation (18) is expressed as the 
difference between the total vertex contribution (J2i fi) and the total edge contribution 
f(ij))- An intuitive understanding of this is as follows: Each edge participates in 
two vertex interactions and its effect is counted twice when calculating the total vertex 
contribution; this over-counting should be subtracted from the total vertex contribution. 

From the viewpoint of partition function loop expansion, the belief-propagation 
fixed-point condition (16) is a requirement for ensuring all the corrections L g from 
subgraphs g with dangling edges are identically zero. For a loopy subgraph g without 
dangling edges, its correction contribution L g is obtained through (13). The correction 
to the total free energy is expressed as the logarithm of the sum of all these loop 
correction contributions [see (17)]. In Appendix A we report the free energy correction 
contribution of a one-dimensional ring of N edges. The correction is found to be positive 
when this ring is energetically frustrated. For more complicated model systems, the sign 
of the free energy correction contribution is still an open issue. 

For a discrete model whose edge states can take Q > 2 different values, Chernyak 
and Chertkov [TT] derived a loop-tower expansion for the partition function by exploiting 
the gauge symmetry of the microscopic states. The derived belief-propagation equation 
by their approach does not fix the gauge freedom completely, and therefore high-order 
gauge fixing was introduced, making the loop-tower expansion scheme very complicated. 
Gauge fixing is not needed in the present loop series expansion scheme. In light 
of the present work, it might be possible to simplify the scheme of [H] and get an 
alternative derivation of the free energy expression (17). We are currently working on 
this mathematical issue. 



4. Graph expansion for the grand partition function 

For the general statistical model defined by the partition function (|2|, the belief- 



propagation equation (16) may have multiple fixed points. If this happens, then the 



BP free energy Fbp as a functional of {pi^j(xij)} has more than one minimal value. In 



the following, we will refer to a fixed-point solution {j>j_^(xy)} of (16) with a minimal 
value of F B p as a macroscopic state of the configuration space. Each macroscopic state 
has a corresponding BP free energy value Fbp- To account for the existence of multiple 
macroscopic states, in analogy with (J5]), we define a grand partition function H as 

S= n[/ / Dft^jD^-tf^fft-tf - £i^j(g«\j))*(^^-%*(gfl!j\i)) exp(-yF BP ). (21) 

(m) 

In the above equation, / Dq^j means summing over all different possibilities of the 
distribution q^j, and the Dirac delta functions S^q^j — B i ^j(qg i \j)^j ensure that only 
fixed-point solutions of the belief-propagation equation ( JT6] ) contribute to H. The 
parameter y is an introduced inverse temperature at the level of macroscopic states. 
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In analogy with ([7]) we can rewrite (21) as 
1 N r 

s = ^ n[n / D^p^(^) e -»/«] n [i + a£ 

In the above equation, P^j(qi^j) is an introduced auxiliary probability distribution 
function for the probability distribution q^jixji)] fa is the free energy contribution of 
vertex % as expressed by (19); C^j^ is an edge constant, 



(22) 



Dry, t; l)f/ ; .,/', > ; (V/, v ) /',- U/ ; .,•)' 



(23) 



A (i) 



with /(j j) being the free energy contribution of an edge (z, j) as given by (20); and 
is expressed as 

(i) = d(g»-*j - Bjqai^Sfa^i - B(q dAi ))C^ ) j) ^ ^ 

~ I', -jiq; •,)') ■;!'/, •;)< y// 

The grand partition function can therefore be expanded as 

= = e sp (i + e 

where E S p is expressed as 



-5P 



nf =a 


Iljeat / Dqj^iPj^i(qj- 










->jDqj->iPi->j (qi-+j)Pj- 







(24) 
(25) 

(26) 



and the correction term Ljp of a subgraph g is expressed as 



4" = n 







■hPj- 










+ p 




e -Ph{{Pj^i}) 



n ^$,iy 

(k,l)eg 



(27) 



Consider a subgraph g which has a leaf vertex % and a dangling edge After 
integrating over the probabilities around vertex i, the correction contribution of this 
subgraph can be expressed as 



4" = n 



n 



ledk 



keg\i Uledk 



j"Dqi^ k Pi^ k (qi^ k ) 



JDpi^ k Pi^ k (p, 



-Pf k ({pi-+ k )} 



n 



aw 

(m,n) 



(m,n)eg\(ij) 

I /' .jiq, .;) / / 1)7, « ; l>7 ; .,(7/ .-)' ?// ' 

where the probability distribution Pj_ > , 7 (g i _^) is calculated by 



x 



(28) 



Uk&di\j 


J ^Qk^iPk^'Mk^i) 


e !lJ[ ■ (S(qi - P>, yj (q di /)) 


Tlkedi\j 


J ^Qk^iPk^iiQk^i) 


g — yfi— >j 



with 







111 



dxijpoixij) J] / dx ik q k ^i(x ik )e ^(*«*) 



(29) 



(30) 



In accordance with the spin-glass literature, we refer (29) as the the survey-propagation 
equation. 
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The expression inside the curly brackets of (28) is identically zero if p-v^g^) = 
Pi^j(<li^j)- Since the auxiliary probability distributions {P^^g^)} are free to choose, 
we can choose them appropriately to ensure that the correction contribution L^' = 
for any a subgraph g with at least one dangling edge. In other words, {p^^g^j)} 



should be a fixed-point solution of the survey-propagation equation: 



Pi— >j {Qi-^j) 



Hk£di\j 


I Dgfc-s>iPe-s>i(gfc-s>i) 


e !lJ[ ■ ()((!, - II, v (g, /; ,)) 


Ukedi\j 


/ Dg/c-5>jPfc-5>j(gfc-^j) 


g — yfi— *j 



This equation was first derived in 
approximation was again used). 



(31) 



under physical considerations (the BP 



At a fixed point of (31 ), the expression of the total grand free energy is 
G(y; 0) = - 1 lm 



y 



G SP (y;P)- -In 

y 



g'QG 



(32) 



where g 1 again denotes a looped subgraph that contains no dangling edges. From the 
framework of partition function loop expansion, (31) is a requirement to ensure that 
subgraphs with dangling edges do not have correction contributions to the grand free 
energy. 



In (32), the grand free energy Gsp(y\ 0) is expressed as 
G S p(y,P) = --lnS 5P = Y^9% ~J29(ij), 



y 



(iJ) 



with 



jedi 



9(i,j) 



In 



-yfi 



-yf(ij) 



(33) 

(34) 
(35) 



being, respectively, the contribution to the grand free energy from a vertex % and an 
edge Gsp(y;(3) is identical in form to the 1RSB free energy of the mean-field 

spin-glass theory [T2] , which was derived previously by applying the BP approximation. 
Gsp can also be regarded as a functional of the 2M probabilities {p^^g^j)}, and 
its variation with respect to any a Pj^j(gi^j) is zero at a fixed-point of the survey- 
propagation equation. We refer Gsp as the survey-propagation free energy functional 
(it is the BP free energy functional at the 1RSB mean-field level). 

We end this section with a discussion on the reweighting parameter y of (21). 
Denoting a fixed-point solution of the belief-propagation equation (a macroscopic state) 
as a and its associated BP free energy as Pgp, the grand partition function H can be 
re-written as 



^exp(-yP^) = / d/exp[tf(E(/) -yf) 



(36) 



where expfiVI](/)j is the total number of macroscopic states with a given BP free 
energy F BP = Nf (the quantity / is called the free energy density). The function £(/) 
is called the complexity in the spin-glass literature (it is the entropy density at the level 
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of macroscopic states). For N ^> 1, the integration in (36) are dominated by the value 



of / = / which satisfies ^p-\f = -j = y- The value / is the mean BP free energy density 



at a given value of y. If we neglect the loop correction to the grand free energy in (32), 
then 

A?- ^^' = E 7. -E (37) 

where f i and f uj\ are, respectively, the mean free energy contribution of a vertex i and 
an edge (i, j), with the expression 

-J _ Tljedi I Dq.j^iPj^i{q.j^>i)fi e V ^ ,„q\ 

,• = / J 1)^ >j\)<lj v (7- >,,)!) *;[<!;, >i)f ■.;..;■/ ;/j ' (3g) 

J{1 ' 3) ~~ 11\)<,; . J \),, J .,(<,, ,,)!) .,[,,, ,)< > ! ' ( -> ' 1 '' 

The value of the complexity S is expressed as 



7-^Gsp(yiP)]. (40) 

The smallest mean free energy density / corresponds to the value of y which makes 
the complexity be zero, S = 0. Another special value of y is y = (5. If the complexity 
calculated at y — /3 is positive, the corresponding mean free energy density value / is 
the typical value of BP free energy densities of the macroscopic states sampled at inverse 
temperature (3 [T2] . 

5. Conclusion and discussion 



The main results of this paper are the free energy expression (17) and the grand 



free energy expression (32), and the corresponding belief-propagation equation (16) 



and survey-propagation equation (31). From the viewpoint of partition function loop 
expansion, the belief-propagation and survey-propagation equation are, respectively, 
conditions needed to ensure that subgraphs with dangling edges have zero correction 
contributions to the free energy and the grand free energy. 

This work helps to place the mean-field RSB theory of spin glasses on a firmer 
mathematical ground. There are many unsolved problems ahead. For example, the 
relationship free energy functionals Gbp and Gsp and the free energy landscape of 
the system; the link between the defined grand partition function 5 and the original 
partition function Z; the issue of sampling microscopic configurations {xy} giving a 
fixed-point solution {qj-n(xij)} of the belief-propagation equation; and son on. 

The discussion in Sec. [4] can be readily extended to the case that the survey- 



propagation equation (31) has multiple fixed-point solutions. As a result, the mean-field 
second-step RSB free energy and its loop correction expression will be derived, as well as 
the corresponding message-passing equation. This expansion hierarchy can be continued 
to produce the mean-field results and the corresponding loop correction expressions and 
message-passing equations at even higher-levels of replica-symmetry-breaking. 
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For statistical models defined on a factor graph with partition functions expressed 
in the form of (HI), the method of this paper can also be directly applied without the 
need of first turning the partition function into the form of §2^. 

The present paper also points to some other important open issues. One question 
is: How to express the mean value of a local observable in terms of a finite loop series? 
Examples of local observables are the state variable Xij on an edge of the system, 
the correlation between two edge variables x^ and x^i, the energy of a single interaction, 
and so on. Loop series expressions for these local observations should be very useful 
in improving the predictions of the mean-field cavity theory. For a graphical model 
with many short loops, it is desirable to represent the system as a collection of many 
basic clusters in the framework of Kikuchi's cluster variation method (for a review, 
see [17J). These basic clusters are connected to each other by joint clusters [TS] . The 
joint clusters can be regarded as generalized edges. The present partition function loop 
expansion method probably is also applicable to these more complex graphical systems. 
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Appendix A. Discrete models on a one-dimensional ring 

As a simple application of the graph expansion method discussed in the main text, we 
calculate the loop correction contribution for a model defined on a one-dimensional ring 
with N vertices and N edges. We assume that the edge state Xij+x between two vertices 
i and (i + 1) can take Q different discrete values, x i>i+ i G {1, 2, ... , Q}. The energy of 
the ring is 

N-l 

8=2 

where J is a coupling constant and 5% is the Kronecker symbol. The prior distribution 
po(xi j i + i) is uniform over the Q states. 



For this simple system, the fixed point of the belief-propagation equation (16) 
is qi-+(i +1 )(x) = 1/Q and q^ i+1 ^i(x) = l/Q for x G {1,2, ... ,Q}. Then we have 
A,- = QSxl^+i — 1, and by a straightforward summation along the ring, 



the loop correction expression (13) is simplified as 



We notice that for N being even, L g > and therefore the loop correction to the free 



energy [see (17)] is negative (the Bethe-Peierls free energy Fgp is higher than the true 
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free energy). However, if iV is odd and J < 0, then L g < and the loop correction to 
the free energy becomes positive (F^p is lower than the true free energy). This different 
behaviour is related to the fact that, the one-dimensional ring with an odd number of 
interactions is frustrated when J < 0. This simple example also shows that the loop 
correction L g decays exponentially with loop length. 
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